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The Data Quality Campaign is a national, collaborative effort to encourage and support state 
policymakers to improve the collection, availability and use of high-quality education data 
and to implement state longitudinal data systems to improve student achievement. The 
campaign aims to provide tools and resources that will assist state development of quality 
longitudinal data systems, while also providing a national forum for reducing duplication of 
effort and promoting greater coordination and consensus among the organizations focusing 
on improving data quality, access and use. 


To these ends, four site visits were conducted in the spring of 2006 to state education 
agencies (SEAs) to gather information on their experiences in developing statewide 
longitudinal data systems: Florida, Utah, Virginia, and Wisconsin. 


This publication was produced by the Data Quality Campaign/National Center for 
Educational Accountability for submission under contract with the Bill & Melinda Gates 
Foundation. The report was written by Cherry Kugle and Nancy Smith and was reviewed and 
approved by the Virginia Department of Education. 


© Copyright 2006. Data Quality Campaign. All rights reserved. 
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Virginia: Enjoying Support 

The Virginia Department of Education (VDOE) oversees 132 public school districts 
(divisions) in which over 1.2 million students were enrolled in the 2005-06 school year. 
Public school students in Virginia participate in the largest on-line assessment program in 
the country. 

History of Development 

Data Collection 

Under the umbrella name of Education Information Management System (El MS), the VDOE 
has set priorities to meet state and federal reporting requirements and enable stakeholders 
at all levels of education to make informed educational decisions based on accurate and 
timely information. The components include: decision support tools for educators at the 
state and local levels; standard and ad-hoc reporting tools for the VDOE; web- based data 
loading and data entry capability; secure data transfer to and from educational entities over 
the Internet; assignment of a unique testing identifier to each student in the state; and 
training for educators on using information to make better decisions. Virginia is adopting 
the Schools I nteroperability Framework (SI F) specification as their standard for transmitting 
data to El MS. The El MS is not intended to be a school or district- level information 
management system for scheduling, grade reporting, or attendance accounting. 

Online testing is an important component of the VDOE's data collection effort and is the 
largest online testing program in the country. Testing takes places in the fall, spring and 
summer, each division setting their own three-week window. The Standards of Learning 
(SOL) for Virginia Public Schools describe the commonwealth's expectations for student 
learning and achievement in grades K-12 in English, mathematics, science, history and 
social science, technology, the fine arts, foreign language, health and physical education 
and driver education. In 2000, the commonwealth launched the Standards of Learning 
(SOL) Technology Initiative with the goal of reducing student-to-computer ratios, creating 
I nternet-ready local area networks and high-speed, high bandwidth capability in all schools, 
and establishing a statewide online testing system. Mandated by the state legislature, a 
major impetus for online testing was faster turnaround of results. The use of online testing 
was important due to the volume of assessments and opportunity for retakes in Virginia. 
According to VDOE staff, paper testing was driving instruction to a crawl. 

The testing contractor maintains the assessment data, with five years of data in the Data 
Warehouse as of the 2005-06 school year. Answer documents are collected on all students, 
tested and untested. Currently, school- level data can be disaggregated to the strand level, 
and student level data are available to teachers from 2005 forward. 

The VDOE began collecting student level enrollment data at the end of the school year 
(EOY) in 2002. The Student Record Collection (SRC) is a snapshot enrollment collected 
multiple times a year [September, March, June, and August with a December count of 
students receiving services in special education being contemplated]. 

Unique Identifiers 

All data collections are electronically submitted via a single sign-on portal (Single Sign-on to 
Web Systems, SSWS). SSWS accounts are managed locally. As much as possible, one 
record layout is used for all data collections, including assessment data. By March 2004, all 
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of the public school students in Virginia had identifying numbers assigned (SSIDs). 

Although this process was not entirely integrated into all of the districts at that time, the 
first records collected with the SSID were at the end of the year in 2005. During the 2005- 
06 school year, the SSIDs were used for state assessment data collection (State Testing 
Identifier, STI ). 

There is a statewide identification (ID) system for teachers in which a unique statewide ID 
(not necessarily a Social Security Number) is assigned to individuals receiving a teaching 
license. Currently, teachers can be connected to the courses they teach, but course 
enrollment or course completion by student is not collected. 

Implementation Issues 

The General Assembly in Virginia is considered to be a leader in the nation in providing 
resources for technology and implementation of student information and data collection 
systems. 

The governor supported the statewide longitudinal student data collection effort because of 
his interest in examining the relationships among teachers' university educations and the 
subsequent performance of their students. Representatives from a consulting firm met with 
the governor to discuss the relevant issues and facilitate political support. The State 
Superintendent supported the effort because of her belief that using data to make decisions 
at the local level is important - that data should be used for informing instruction as well as 
for evaluating schools and districts. 

Funding was provided by the state to support the effort. Although staff had planned to take 
a longer time to implement the system, in order to complete the process within the 
governor's term the VDOE spent one year and three months from the time of awarding the 
contract to the completion of the assignment. The Request for Proposal was written in 
August, awarded in December, and work started in March. 

A student information advisory group was created to provide reality checks and obtain 
district buy-in. Ownership of the project was fostered by district representatives being 
included in and shaping the process. 

Virginia was an early participant in the Schools Interoperability Framework (SIF), a non- 
profit technical standards association that collaborates on developing standards for sharing 
education data quickly and securely. It was the first state to purchase SIF memberships for 
each of its districts. 

The State Board of Education in Virginia is the state education agency and is the entity that 
has responsibility for federal reporting. Time for board review and approval must be 
factored into all calendars and scheduling. 

Costs 

Estimated costs to the state: 

o The cost of the statewide student information system and assessment data 

warehouse is approximately $3 million per year, plus the cost of staff time at the 
VDOE. 

o Beginning in 2000, from a $360 million appropriation, the Standards of Learning 
(SOL) Technology Initiative provided funds to schools to build infrastructure to 
reduce student-to-computer ratios, create I nternet-ready local area networks and 
high-speed, high bandwidth capability in all schools, and establish a statewide online 
testing system. 
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Estimated costs to districts: 

o School division data system personnel are not currently funded by the state. 

o The costs for data submission are rarely funded directly; however, schools are given 
money from the state to implement the Standards of Quality and they may choose to 
use the money on behalf of data collection and submission. 

o Districts were encouraged to "make friends" with the federal program offices where 
funds might be available. 

o VDOE negotiated a very low price for divisions to expand their use of SI F beyond 
state reporting. 

Benefits and Uses of System 

Benefits 

o Data integrity has improved. Data are much more likely to be accurate since they 
are coming from one source, with the addition of assessment variables added at the 
time of testing. 

o Having a consistent record layout for the collections has helped with submitting and 
extracting data. 

o The unique identifier enables the state to account for every student. 

o Online testing is much more efficient. 

Uses 

o The online assessment system provides an introduction to electronic reports for 
many users. Rather than receiving printed reports from the state or assessment 
contractor, educators can request what is needed and print it locally, or archive an 
electronic copy. These changes were somewhat painful at first but are now widely 
perceived as useful and more efficient. 

o Data are checked using statistical procedures to look for unexpected values and large 
differences from data submitted the prior year. Verification reports are sent to the 
districts for their review and possible action. Once the data are deemed final by the 
district, the reports are signed by the district superintendent or director of finance. 

o The VDOE does not currently provide student- level data, even redacted, to anyone. 
Researchers may obtain aggregate level data with small cell sizes (N <= 10) 
suppressed. 

o Once the data become available, requests for information will follow. VDOE staff 
recommend anticipating this by creating formats early in the process that will allow 
the data to be accessible and reduce the number of customized extractions required 
of staff. Consider creating a website with public query tools or forming a partnership 
with another organization to provide the data following the state department's 
requirements for maintaining security and confidentiality. 

Lessons Learned 

Design 

o Keep your scope small and have clear deliverables. 

o Expect the process to be extremely painful for both districts and the state as they 
work together to get the kinks out of the system the first year or two. 

o Develop a long-term view. The VDOE has a ten-year plan which is implemented two 
years at a time due to the legislative process. 

o Understand that the reality of implementation will be somewhat fragmented. 
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Maintenance/ Change Control 

o Provide the flexibility to meet the diverse needs of teachers, principals, 
superintendents, school board members and other policymakers. 

o Creating a set of standards for entering names before the system is implemented 
can improve the quality of data and help resolve matching of students. 

o Publish the data quickly and regularly to improve accuracy. 

o Emphasize that policy decisions and instructional decisions are based on data so they 
have to be the best they can be. 

Partnerships 

Vendor 

o When writing a Request for Proposal (RFP), be as specific in delineating your 

requirements as possible, and try to have clear expectations between the state and 
the vendor. 

o Be knowledgeable about what you are looking for and have the vendor demonstrate 
it for you. If you do not have in-house experience, go visit states that do and learn 
from them. 

o Ask the respondents how they would evaluate themselves regarding their success in 
completing the task. 

o Encourage the vendors to be creative in their proposals. Put an open-ended scenario 
in the RFP and assess the responses. Look for a good mix of specificity, flexibility, 
and creativity. 

o In reviewing the responses to RFPs, "beware the snake oil salesman." A response of 
"32,000 hits on the website" versus "principals using the reports to inform decisions" 
is very telling. 

o A good RFP response should demonstrate an understanding of education data and 
the current state of education data systems in the world today. Responses that are 
overly technical are not the best, nor are well-known "big name" companies 
necessarily the best ones to do the job. What is required is an understanding of 
education data collection, submission, and extraction processes from the local level 
to the state level. 

o Look for a vendor who knows more about what you need than you do. 

o Work with vendors to identify what is needed at the local level as well as what the 
state needs. 


District 

o Work to help local administrators and teachers become better consumers of the data, 
so that they know how to use them and are not uncomfortable accessing and 
reviewing reports. 

o Nurture the relationships between the people managing the student data and those 
managing the assessment data at the local level. Both groups need to understand 
all the components of the data system and how they fit together, 
o Try to give districts as much notice as possible for changes and additions to data 
collection. A good goal is to give districts at least one year of notice as data may 
need to be collected throughout a full school year, 
o Districts should realize that a statewide data tracking system is not going to replace 
everything that is needed at the local level. "Don't give up on local richness of data 
but make sure you are consistent in providing good data to the state." 
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Staffing 

o Create a panel to evaluate the responses to the RFP. If possible, include a local 

assessment director, the state assessment director, IT staff from within the state and 
from another state, and more educators than IT staff. 

o Require experienced staff to be on your vendor team - those who can help shape the 
products when the state doesn't yet know the answers to all the questions about 
what they need. 

o Cultivate a highly-placed person with credibility to help with the policymakers and 
bring their understanding of the system along. It is great to have someone in a high 
position support you and help fill in the gaps of understanding at the upper levels. 

o It is extremely helpful for managers to allow staff to take leadership and to pursue 
different avenues as needed. For example, staff needed to know how principals use 
data and what training they needed and were given the freedom to put a study 
together to answer those questions. 

o Try to put a mix of advocates and dissenters on the advisory board so you hear all 
the issues up front and can address them. 

Recommendations for Future Development 

o Provide "user friendly" access to data for all interested parties - from teachers to the 
chief state school officer - to reduce the number of ad hoc requests currently being 
filled by VDOE staff. 

o To provide continuity over time in content of reports, document the data that need to 
be compiled for each report. 

o I mplement a student-level course completion data collection to complete the link 
between teachers, courses, and students. 

o Implement an electronic transcript concept to facilitate moving student information 
between schools and from P-12 to higher education. 

o I mplement collection of student- level SAT, ACT and AP data. 

o Provide resources to teachers that connect data on student achievement with 
strategies that can be used to improve instruction. 

o Continue to implement the SIF standards. 
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